OvarianCancer-NCI-PBSII-061902

Emanuel F Petricoin III, et al. "Use of Proteomic Patterns in Serum to Identify Ovarian Cancer". The Lancet, 359:572-577, February, 2002

Generated by surface-enhanced laser desorption and ionisation mass spectroscopy. SELDI—a refinement of MALDI—preselects the proteins in the sample by allowing them to bind to the treated surface of a metal bar, which is coated with a specific chemical that binds a subset of the proteins within the serum sample

The goal of this experiment is to identify proteomic patterns in serum that distinguish ovarian cancer from non-cancer. This study is significant to women who have a high risk of ovarian cancer due to family or personal history of cancer. The proteomic spectra were generated by mass spectroscopy and the data set provided here is 6-19-02, which includes 91 controls (Normal) and 162 ovarian cancers. The raw spectral data of each sample contains the relative amplitude of the intensity at each molecular mass / charge (M/Z) identity. There are total 15154 M/Z identities. The intensity values were normalized according to the formula: NV = (V-Min)/(Max-Min), where NV is the normalized value, V the raw value, Min the minimum intensity and Max the maximum intensity. The normalization is done over all the 253 samples for all 15154 M/Z identities. After the normalization, each intensity value is to fall within the range of 0 to 1.

The original dataset is in hdf5 file (can open in HDFView). We have converted to a CSV file.

Below is the code to convert hdf5 to csv:

    import h5py

    path = os.getcwd() + '/ovarian-cancer-nci-pbsii-data.h5'
    f = h5py.File(path, 'r')

    features = []
    for item in f['/data_descr/ordering']:
        features.append(item.decode("utf-8"))
        
    X = np.zeros((253,0))
    for feature in features:
        a = f['/data/'+feature].value.reshape(-1,253).T
        #print(feature + ":", a.shape)
        X = np.hstack((X,a))

    yb = X[:,-1] # last column is y label
    ys = []
    for item in yb:
        ys.append(item.decode("utf-8"))
    y = []
    label_names = list(set(ys)) # get distinct y labels
    for item in ys:
        y.append(label_names.index(item)) # 将y字符串标签转化为int
    labels = np.arange(len(label_names))

    X = X[:,:-1]

    featureNames = []
    for item in f['/data_descr/names'][:-1]: # the last column name is 'Class' (y), skip it
        # featureNames.append(float(item.decode("utf-8").replace('MZ','')))
        featureNames.append(item.decode("utf-8"))

    print("X = {}, y = {}".format(X.shape, len(ys)))
    print("label names = {}, label codes = {}".format(label_names, labels))

    X_names = list(map(lambda x: float(x.replace('MZ', '')), featureNames))

    import pandas as pd
    df = pd.DataFrame(X,columns=X_names)
    df.insert(0, 'Class', y)

    # write to a csv file
    df.to_csv('ovarian-cancer-nci-pbsii-data.csv')